Evaluating one-sided programming models for GPU cluster computations

The Global Array toolkit (GA) [1] is a powerful framework for implementing algorithms with irregular communication patterns, such as those of quantum chemistry. On the other hand, accelerators such as GPUs have shown great potential for important kernels in quantum chemistry, for example, atomic integral generation [2] and dense linear algebra in correlated methods [3]. Integration of the global address space (GAS) programming model and associated one-sided protocols with GPU programming paradigms such as CUDA has the potential to revolutionize quantum chemistry by allowing the efficient use of very large clusters of heterogeneous nodes, such as the future multi-petaflop installation expected at Oak Ridge National Laboratory in 2012. This paper reports on our preliminary investigations of the technical challenges and performance opportunities associated with cluster-GPU computation using the simplest approximation to quantum chemistry applications: parallel matrix-matrix multiplication (MMM). We focus on the role of asynchronous execution of network communication, device-to-host transfer, and kernel launch to understand the extent of latency-hiding that can be achieved for dense algorithms on large matrices.